Repository navigation
fix(schema): keep structured output JSON serializable - #7591
BlueRaddish wants to merge 3 commits into
Conversation
Serialize validated model output in JSON mode so date-valued AgentTool results survive the real Runner and tool-result serialization. Keep the existing exclusion of None fields for model and list output schemas. Related to google#7480. AI-assisted with OpenAI Codex.
Somay-kousis
left a comment
Somay-kousis
left a comment
There was a problem hiding this comment.
Checked the three new tests against the base (a01591c78~1) and the head: test_structured_output_preserves_json_date[False], [True] and test_agent_tool_structured_date_output_survives_anthropic_conversion fail on the base and pass on the head.
One sibling path still has the bug. When an agent has both output_schema and tools and its model can't do both natively (capabilities.output_schema_and_tools is False, which is the case for claude-* names and the LiteLlm providers listed in lite_llm.py), the final answer goes through SetModelResponseTool.run_async, not validate_schema. That tool still dumps in Python mode (set_model_response_tool.py:304 and :313), and get_structured_model_response then does json.dumps(response) (flows/llm_flows/prompt/_schema.py:120). So a date field makes the run raise, on any model, before AgentTool or the Anthropic converter come into it.
Repro: one LlmAgent with output_schema=DatedOutput (day: date), output_key="out", a fake model, InMemoryRunner. With tools=[lookup] the model answers by calling set_model_response with {"day": "2026-10-17"}.
On this branch:
--- output_schema=DatedOutput, tools=[]
capabilities.output_schema_and_tools = False
final text: {"day": "2026-10-17"}
state out: {'day': '2026-10-17'}
--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
run raised: TypeError: Object of type date is not JSON serializable
Adding mode="json" to the two model_dump calls in SetModelResponseTool.run_async fixes it (I tried it on your branch and reverted):
--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
final text: {"day": "2026-10-17"}
state out: {'day': '2026-10-17'}
With that change tests/unittests/tools/test_set_model_response_tool.py, tests/unittests/utils, tests/unittests/tools/test_agent_tool.py and tests/unittests/flows give 1900 passed. Might be worth folding in here with a test in test_set_model_response_tool.py, since it's the same dump and the same #7480 case.
Probe script: probe_7591.py (outputs probe_7591_head.out, probe_7591_base.out, probe_7591_suggested.out).
Serialize validated BaseModel output and list items in JSON mode before set_model_response enters event and final-response JSON serialization. Preserve validation feedback and omission of None fields. Refs google#7480 AI-assisted with OpenAI Codex.
Resolve the validate_schema conflict by retaining upstream Annotated unwrapping and type narrowing together with JSON-mode output. Cover annotated scalar, list item and outer list dates and omitted None fields. Refs google#7480 AI-assisted with OpenAI Codex.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Related to #7480. Pydantic validation can turn JSON date strings into Python
dateobjects. Dumping those models in Python mode leaves non-JSON values in structured output and AgentTool responses. Theset_model_responsefallback used when tools cannot be combined with an output schema has the same problem, identified in this review.Use JSON-mode model dumping for single models and list items in both
validate_schemaandSetModelResponseTool.run_async. Retainexclude_none=True, validation feedback and primitive/non-model behavior. Schema declarations, APIs, dependencies and providers are unchanged.This update merges current main
37b1ab3c20a0bee18e5c4429a05a0007cd8ea8d6and resolves thevalidate_schemaconflict by preserving upstream Annotated unwrapping, BaseModel type narrowing and the typed list adapter together with JSON-mode dumping. Three additional cases cover an annotated model, an annotated list item and an annotated outer list, including omittedNonefields.Testing Plan
The exact normally-hooked two-parent merge commit
439a9d1190aab813146f714c2ec2c5a5da33f2dcpasses the unchanged full supported tox matrix:Each lane also verifies the real installed workspace import, source/configuration hashes and lock preservation; passes 1,975 owning schema/tool/flow tests with zero skips; and passes ordinary changed-file hooks plus the actual Git pre-commit hook. All four jobs produce the same raw native commit, tree and two parents. Full-tox skips, xfails, xpasses and warnings remain visible in the logs.
The current integration regression checks use unchanged production baselines:
bb7472c3fa73aa305a662381b63bca965be56956, substituted into this integration checkout: all seven cases pass. This confirms the merge preserves the existing fix; Annotated support is not presented as a defect in that published head.TypeError: Object of type date is not JSON serializableagainsta01591c78332256ca14579184f4e9091779d3851, while two no-tools controls pass. All four pass with the fallback fix. They assertoutput_schema_and_tools=False, actual injected-tool dispatch, final JSON andoutput_keysession state. Historical fallback verification.a6ed85905d4c1d1f285ba9d4f76a9d801b66d17f, including a real Runner/AgentTool response through the public Anthropic converter. Historical verification.Unit Tests:
Manual End-to-End (E2E) Tests:
The consuming tests use the repository's scripted model with its real InMemoryRunner, tools and session state. No live model/provider, browser UI or manual cloud acceptance test was run.
Checklist
Additional context
The first merge validation passed the regression and 1,975 owning cases on all four Python versions, then failed the ordinary formatter hook on one added Annotated test expression. Its identical actual formatter diff was applied; test AST and production bytes were unchanged. That failed attempt is retained in the earlier run, followed by the complete successful matrix above.
Matching open proposals were checked for supersession. The separate non-model validation proposal #7285 leaves BaseModel output dumps unchanged; schema-declaration serialization #7626 changes a different conversion path. No overlapping current proposal repairs the fallback tool's model-value serialization.
OpenAI Codex assisted with diagnosis, implementation, deterministic verification and this description. No full live-service or model evaluation is claimed.